Analysis device, microscope device, analysis method, and program

ABSTRACT

An analysis device for quantifying a state of a fluorescent image containing a plurality of bright spots comprises an area setting unit in which states of a plurality of bright spots contained in a plurality of areas set in the fluorescent image in accordance with positions of the plurality of bright spots are quantified as numerical values.

This is a Continuation Application of U.S. patent application Ser. No.15/079,568 filed on Mar. 24, 2016 which is a Continuation Application ofInternational Application No. PCT/JP2014/075658, filed on Sep. 26, 2014,which claims priority to Japanese Patent Application No. 2013-200705,filed on Sep. 27, 2013, the contents of the above applications beinghereby incorporated by reference.

BACKGROUND Technical Field

The present invention relates to an analysis device, a microscopedevice, an analysis method, and a program.

The present application claims priority to Japanese Patent ApplicationNo. 2013-200705, filed Sep. 27, 2013, the content of which isincorporated herein by reference.

Background Art

Super-resolution microscopes are microscope devices which usefluorescence techniques to allow for observation beyond the resolutionof an optical system. One embodiment of known super-resolutionmicroscopy is stochastic optical reconstruction microscopy (STORM; e.g.see Patent Document 1). In STORM, a fluorescent material or object withsaid fluorescent material adhered thereto is used as an observationsample. This fluorescent material has properties whereby it becomesactive when irradiated with activation light of a predeterminedwavelength and, thereafter, becomes inactive by emitting fluorescencewhen irradiated with excitation light of a wavelength different from theactivation light. A fluorescent image is acquired by irradiating theobservation sample with activation light at a low power so as toactivate the fluorescent material at a low density and then applyingexcitation light so as to cause the fluorescent material to emit light.In a fluorescent image acquired in this manner, fluorescent bright spots(images of the fluorescent material) are distributed at low density andare individually isolated. As a result, the center-of-gravity positionof each individual image can be determined. This step of acquiring afluorescent image is repeated multiple times, e.g. hundreds to tens ofthousands of times, and the resulting fluorescent images are synthesizedthrough image processing to enable the generation of a sample pictureimage of high resolution.

In STORM, a technique for determining the locations of the fluorescentbright spots is known in which the locations of the fluorescent brightspots are pseudo-calculated from the results of calculating theprobability distribution based on obtained data (Gaussian distribution)(e.g. see Non-Patent Document 1).

CITATION LIST Patent Literature

-   Patent Document 1: U.S. Patent Application Publication No.    2008/0032414

Non Patent Literature

-   Non-Patent Document 1: Sara A Jones, Sang-Hee Shim, Jiang He &    Xiaowei Zhuang, “Fast, three-dimensional super-resolution imaging of    live cells”, Nature America, Inc., 2011

SUMMARY Technical Problem

However, while it is possible to acquire fluorescent images at highresolution using STORM, the bright spot data (e.g. number of photons,ellipticity, etc.) are localized to particular coordinates (e.g.center-of-gravity position, etc.). As a result, there is a problem inthat quantitative analysis such as that including basic arithmeticoperations using the pixel intensity value (quantity of fluorescentlight) of the image (or between images) cannot be performed.

An object of the aspects according to the present invention is toprovide an analysis device capable of performing quantitative analysisbased on images of high resolution, a microscope device, an analysismethod, and a program.

Solution to Problem

An analysis device according to an aspect of the present invention is ananalysis device for quantifying a state of a fluorescent imagecontaining a plurality of bright spots. The analysis device comprises anarea setting unit in which states of the plurality of bright spotscontained in a plurality of areas set in the fluorescent image inaccordance with positions of the plurality of bright spots arequantified as numerical values.

A microscope device according to an aspect of the present inventioncomprises a sample picture image generating device that generates afluorescent image containing the plurality of bright spots by displayingthe positions of the images of a fluorescent material contained in theacquired fluorescent image as bright spots.

An analysis method of an aspect according to the present invention is ananalysis method for quantifying a state of a fluorescent imagecontaining a plurality of bright spots. The analysis method comprises anarea setting process in which states of the bright spots contained in aplurality of areas set in the fluorescent image in accordance withpositions of the plurality of bright spots are quantified as numericalvalues.

An aspect according to the present invention is a program for allowing acomputer of an analysis device for quantifying a state of a fluorescentimage containing a plurality of bright spots to execute an area settingstep in which states of bright spots contained in a plurality of areasset in the fluorescent image in accordance with positions of theplurality of bright spots are quantified as numerical values.

An analysis device of an aspect according to the present invention is ananalysis device for quantifying a state of a fluorescent imagecontaining a plurality of bright spots. The analysis device comprises anarea setting unit in which states of the bright spots contained in eacharea of a plurality of preset areas set in the fluorescent image arequantified as numerical values.

Advantageous Effects of Invention

According to the aspects of the present invention, quantitative analysisbased on images of high resolution can be performed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a configuration diagram illustrating a first embodiment of amicroscope device.

FIG. 2 is a schematic drawing of total reflected illumination.

FIG. 3 is a flowchart showing an observation method according to thefirst embodiment.

FIG. 4 is a drawing comparing a conventional image and a STORM image.

FIG. 5 is a schematic drawing conceptually illustrating a conventionalimage.

FIG. 6 is a schematic drawing conceptually illustrating a STORM image.

FIG. 7 is a drawing illustrating bright spots in a STORM image.

FIG. 8 is a drawing illustrating a method of setting areas correspondingto bright spots in a STORM image in the first embodiment.

FIG. 9 is a drawing illustrating a method of setting areas correspondingto bright spots in a STORM image in a second embodiment.

FIG. 10 is a drawing illustrating a method of setting areascorresponding to bright spots in a STORM image in a third embodiment.

FIG. 11 is a drawing illustrating a method of setting areascorresponding to bright spots in a STORM image in a fourth embodiment.

FIG. 12 is a drawing illustrating a method of setting areascorresponding to bright spots in a STORM image in a fifth embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of a microscope device and an imagegeneration method will be described while referring to the drawings.

First, a summary will be presented of an analysis method forquantitatively analyzing a subject having movement similar to that of aliving cell.

For example, analysis techniques exist for quantitatively analyzing thestate of a cell from fluorescent images that have been observedcontinuously. In conventional analysis techniques, acquired fluorescentimages are used with a pixel of an optical receiving element beingtreated as the smallest unit of space. Thus, the intensity distributiondata of the fluorescent image (intensity distribution and the like offluorescent light emitted from the fluorescent material contained in theobservation sample) is allocated at the “pixel” smallest unit of space,and basic arithmetic operations using intensity values (fluorescentlight quantity) of the pixels (or between the pixels), or, in otherwords, basic arithmetic operations using the pixel intensity values ofthe images (or between the images) can be quantitatively performed.Examples of methods of quantitative analysis based on data acquired inthis manner include Ratio (also referred to as “P-B ratio” or “peak tobackground ratio”), fluorescence recovery after photo bleaching (FRAP),fluorescence resonance energy transfer (FRET), and the like. Details onthe aforementioned methods can be found in the following:

-   (1) Rajesh Babu Sekar and Ammasi Periasamy, “Fluorescence resonance    energy transfer (FRET) microscopy imaging of live cell protein    localizations”, The Journal of Cell Biology, Volume 160, Number 5,    Mar. 3, 2003:629-633-   (2) Richard N. Day and Fred Schaufele, “Imaging Molecular    Interactions in Living Cells”, Endocrine Society, Molecular    Endocrinology, July 2005, 19(7):1675-1686-   (3) Peter F. Davies, Jenny Zilberberg, Brian P. Helmke, “Spatial    Microstimuli in Endothelial Mechanosignaling”, American Heart    Association, Journal of the American Heart Association, Circulation    Research 2003; 92:359-370

However, with STORM images generated on the basis of positioninformation of the fluorescent material contained in the observationsample, the bright spot information (number of photons, ellipticity, andthe like) belongs to the position information (center-of-gravitycoordinates). Therefore, the conventional method of quantitativeanalysis cannot be applied as-is to fluorescent images in which theintensity value (fluorescent light quantity) is acquired at the pixellevel.

To answer this problem, an analysis method which allows thequantification of the states of fluorescent images containing aplurality of bright spots and the carrying out of quantitative analysisin sample picture images of high resolution is described in thetechniques shown in the embodiments below.

In the following descriptions, the super-resolution microscope is amicroscope that uses stochastic optical reconstruction microscopy(STORM).

In generally known STORM technology, it is possible to acquire a samplepicture image of high resolution, but the sample picture image isgenerated on the basis of fluorescent bright spots in the plurality offluorescent images detected discretely on the time axis, and a pluralityof fluorescent images detected within a predetermined period of time areneeded in order to acquire one sample picture image.

In STORM, positions of the fluorescent bright spots are computed bycalculating the luminosity distribution in the fluorescent image.

In other words, STORM computes the positions of the fluorescent brightspots on the basis of data obtained optically and increases theresolution at which the positions of the fluorescent bright spots areshown.

As a consequence, in typical STORM, in order to acquire one samplepicture image, the positions of the fluorescent bright spots arecomputed from a plurality of fluorescent images detected at differentpoints of time. With regards to the computation of the positions of thefluorescent bright spots, it is known that the resolution of theacquired sample picture image increases when the number of fluorescentimages is increased. However, in STORM types that probabilisticallycompute the positions of the fluorescent bright spots on the basis ofluminosity distribution, when the number of fluorescent images isincreased in order to increase the resolution, the time necessary toacquire one sample picture image increases and the computation load alsoincreases. On the other hand, when the number of fluorescent images isreduced in an effort to shorten the interval between acquired samplepicture images, it becomes impossible to attain the necessaryresolution.

A technique is described hereinafter that enables the performance ofquantitative analysis on sample picture images of high resolution, whileovercoming the problems associated with generally known STORM types asdescribed above.

In the following analysis method for sample picture images of highresolution, a plurality of fluorescent images containing bright spotscaused by irradiating a sample with excitation light are acquired, andanalysis is carried out on the basis of sample picture images that showthe positions of the bright spots computed from image informationshowing the fluorescent image. Additionally, in this analysis method, onthe basis of the position information computed as the positions ofbright spots in the sample picture image, predetermined areas thatcontain the positions of said bright spots are set as areascorresponding to said bright spots.

Note that the present embodiment is intended to explain the summary ofthe invention in detail so that it can be better understood, and doesnot limit the present invention unless otherwise specified. In addition,in the drawings used in the following explanation, main sections may beshown in an enlarged manner for the sake of convenience so that thefeatures can be easily understood. Accordingly, the scale or the like ofeach requested element is not necessarily the same as the actual scale.

FIG. 1 is a configuration diagram illustrating a microscope deviceaccording to the present embodiment.

The microscope device 10 includes a light source 12, a control unit 14,a microscope main body 15, a storage unit 16, and a display unit 17.

In the present embodiment, the microscope device 10 is a microscopedevice that uses super-resolution microscopy (Stochastic OpticalReconstruction Microscopy: STORM). In the microscope device 10, a samplelabeled with a fluorescent material is used. This fluorescent materialemits fluorescence and becomes inactive upon irradiation with excitationlight L1 while in an active state. The fluorescent material hasproperties by which, after emitting fluorescence upon irradiation withthe excitation light L1 and becoming inactive, the fluorescent materialbecomes reactivated upon irradiation with activation light L2 of awavelength that differs from that of the excitation light L1. Using theexcitation light L1 and the activation light L2, operations are repeatedwherein discretely distributed fluorescence is observed by causing lightemission of only a portion of the fluorescent material in the sample. Asa result, a sample picture image is formed using the numerous acquiredfluorescent images.

The light source 12 according to the present embodiment includes anexcitation illumination system 11 and an activation illumination system13.

The excitation illumination system 11 is provided with a laser lightsource 21, a shutter 22, and a total reflection mirror 32. Theexcitation illumination system 11 is connected to the microscope mainbody 15 via the total reflection mirror 32.

The laser light source 21 is a light source which supplies theexcitation light L1 to the microscope main body 15 for the purpose ofcausing the fluorescent material adhered to the sample to emit light. Itis sufficient that the laser light source 21 emit the excitation lightL1 of a wavelength adapted to the fluorescent material contained in thesample. For example, depending on the type of fluorescent material, agreen laser (wavelength: 532 nm), a red laser (wavelength: 633 nm, 657nm), a violet laser (wavelength: 405 nm), a blue laser (wavelength: 457nm), or the like may be used.

The shutter 22 is a device which conducts changeover between supply andstoppage of the excitation light L1 to the microscope main body 15. Forexample, the shutter 22 may be configured to be provided with a lightshielding member that blocks the excitation light L1 emitted from thelaser light source 21, and a drive apparatus that advances and retractsthe light shielding member relative to and from the optical path of theexcitation light L1.

Alternatively, an acousto-optic tunable filter (AOTF) may be used as theshutter 22. The total reflection mirror 32 serves to totally reflect theexcitation light L1 radiated from the laser light source 21 toward astage 31 (described hereinafter) of the microscope main body 15.

Based on the description above, the excitation illumination system 11 isconfigured to radiate the excitation light L1 to all areas of theobservation view field (observation area) on the stage 31.

On the other hand, the activation illumination system 13 is providedwith a laser light source 42, a scanner 43, and a dichroic mirror 33.The activation illumination system 13 is connected to the microscopemain body 15 via the dichroic mirror 33 being inserted on the opticalpath of the excitation light L1. The dichroic mirror 33 serves toreflect the activation light L2 radiated from the laser light source 42toward the stage 31 and transmit the excitation light L1 toward thestage 31.

The laser light source 42 radiates the activation light L2 toward themicroscope main body 15 for purposes of activating the fluorescentmaterial. It is sufficient that the laser light source 42 emit theactivation light L2 of a wavelength adapted to the fluorescent materialcontained in the sample. For example, depending on the type fluorescentmaterial, a green laser (wavelength: 532 nm), a red laser (wavelength:633 nm, 657 nm), a violet laser (wavelength: 405 nm), a blue laser(wavelength: 457 nm), or the like may be used.

The scanner 43 scans the activation light L2 on the stage 31 of themicroscope main body 15. For example, a biaxial galvano scanner may beused as the scanner 43. Based on the description above, the activationillumination system 13 is configured to enable irradiation of theobservation view field (observation area) on the stage 31 with theactivation light L2 while scanning is conducted by the scanner 43.

Note that a laser light source apparatus provided with the laser lightsource 21 and the laser light source 42 within a single housing, andconfigured to be capable of radiating multiple types of laser light maybe used as the light source 12. In the case where this type of laserlight source apparatus is provided, both the excitation light L1 and theactivation light L2 can be supplied to the microscope main body 15 froma single illumination system by configuring an illumination system thatis provided with the shutter 22 and the scanner 43 along with the laserlight source apparatus.

The microscope main body 15 may, for example, be configured from aninverted microscope. The microscope main body 15 is provided with thestage 31 on which the observation target, or sample, is placed.Additionally, a camera 34 for photographing fluorescent images of thesample placed on the stage 31 is connected to the microscope main body15. The camera 34 may, for example, be a CCD camera having numerouspixels.

Although not illustrated in the drawings, the microscope main body 15 isalso provided with an objective lens that irradiates the stage 31 withthe excitation light L1 and the activation light L2, an image forminglens that couples the fluorescence (observation light) emitted from thefluorescent material in the sample to a light receiving surface of thecamera 34, and the like. The objective lens and the image forming lensdescribed above are configured as the image-forming optical system ofthe present invention by which the sample placed on the stage 31 isobserved.

The stage 31 is configured to enable total reflection illumination thatcauses total reflection of the excitation light L1 and the activationlight L2 at the interface of the sample and a cover glass affixed to thesample.

FIG. 2 is an explanatory drawing illustrating a case where a sample S isirradiated with the activation light L2 (or the excitation light L1) viatotal reflection illumination. As illustrated in FIG. 2, as a result oftotal reflection illumination, the sample S can be illuminated by theevanescent light EV that exudes from the cover glass 31 a to the sampleside when the illumination light (the excitation light L1 or theactivation light L2) is totally reflected. As the range that evanescentlight travels is limited to a range on the order of 100 to 150 nm fromthe interface, only the fluorescent material positioned in the vicinityof the cover glass 31 a surface can be induced to emit light, and thiscan be achieved with a high S/N ratio by markedly mitigating backgroundfluorescence.

The microscope main body 15 of the present embodiments has aconfiguration that enables use by switching between the total reflectionillumination described above and epi-illumination.

Returning to FIG. 1, the control unit 14 described above is a computerthat comprehensibly controls the microscope device 10, and is connectedto the storage unit 16, the display unit 17, and a camera controller 19.In the present embodiments, the control unit 14 has at least a controlsignal generation function that generates control signals for purposesof controlling these devices, a fluorescent image acquisition unit 141that acquires fluorescent images via the camera controller 19, a pictureimage forming unit 142 that generates a sample picture image from aplurality of fluorescent images, and a sample picture image analysisunit 143 that performs an analysis based on the generated sample pictureimage.

Additionally, the sample picture image analysis unit 143 is providedwith an area setting unit 143A and an analysis processing unit 143B. Thearea setting unit 143A sets predetermined areas relative to the brightspots on the basis of the sample picture image generated by the pictureimage forming unit 142. The analysis processing unit 143B performsquantitative analysis processing based on the image informationrepresenting the predetermined areas set at each of the bright spots bythe area setting unit 143A. Ratio or FRAP, for example, may be includedas the quantitative analysis processing. Next, a description of thespecific processing carried out by sample picture image analysis unit143 shall be given.

The storage unit 16 is composed, for example, of a semiconductor memory,a hard disk, or the like, and stores a program used in the control unit14 and data (fluorescent images and the like) supplied from the controlunit 14 in a manner that allows read-out from the control unit 14. Notethat when performing the analysis processing of the present embodiments,for each of the bright spots, the storage unit 16 associates and storesposition information showing the position of the bright spot andinformation showing the state of the bright spot (numerical value or thelike) with identification information that identifies the bright spot,and also stores information showing the area associated with the brightspot.

The display unit 17 is, for example, a monitor (display device) or aprinter (printing device), and provides functions for displaying and/orprinting images based on the picture image data outputted from thecontrol unit 14. In the present embodiments, the display unit 17 is amonitor.

The camera controller 19 conducts drive control of the camera 34connected to the microscope main body 15.

The camera controller 19 operates the camera 34 on the basis of controlsignals inputted from the control unit 14, acquires picture images ofthe fluorescence radiated from the sample, and outputs the acquiredfluorescent images to the control unit 14.

The microscope device 10 implements the various types of operationswhich are required to execute the image processing method describedbelow by performing in combination the functions provided to the controlunit 14 described above. Accordingly, the microscope device 10 isadditionally provided with a sample picture image generating device thatgenerates a fluorescent image through STORM photograph processing andimage processing, specifically by generating a fluorescent imagecontaining the plurality of bright spots by displaying the positions ofthe images of the fluorescent material contained in the acquiredfluorescent image as bright spots; and a sample picture image analysisdevice that performs an analysis based on the generated sample pictureimage.

Next a description is given of an example of the picture image analysismethod of the present invention, on the basis of a description of theoperation of the microscope device 10. The microscope device 10 performspicture image analysis on captured sample picture images usingsuper-resolution microscopy technology.

FIG. 3 is a flowchart showing the image processing method of the presentembodiment. Hereinafter, the image processing method according to thepresent embodiment will also be described within the context of thedescription of the image observation method of the microscope device 10.

The image processing method is composed of an initialization processstep S101, a sample picture image generation step S102, and a samplepicture image analysis step S103.

The initialization process step S101 includes a step S11 in whichinitialization processing required for later observation processing withthe microscope device 10 is performed. Additionally, the sample pictureimage generation step S102 includes a step S12 in which the observationview field is irradiated with the activation light L2; a step S13 inwhich, following irradiation with the activation light L2, theobservation view field is irradiated with the excitation light L1 and asecond fluorescent image is acquired; a step S14 in which the secondfluorescent image is saved; a step S15 in which capture completion isdetermined; and a step S16 in which a sample picture image is generatedfrom a plurality of the second fluorescent images.

Additionally, the sample picture image analysis step S103 includes astep S17 in which an analysis based on the STORM image (sample pictureimage) is performed and, on the basis of the position informationcomputed as the positions of bright spots in the STORM image,predetermined areas that contain the positions of said bright spots areset as areas corresponding to said bright spots.

A summary of the image observation procedure using the microscope device10 is given below.

First, in the initialization process step S101, initializationprocessing is completed. Thereafter, in the sample picture imagegeneration step S102, an operation of irradiating the sample with theactivation light L2 and an operation of irradiating the sample with theexcitation light L1 to acquire the second fluorescent image are repeatedhundreds to tens-of-thousands of times (STORM capture processing). Then,a STORM image of high resolution is acquired by synthesizing themultiple second fluorescent images that were captured.

The difference between images observed without the use ofsuper-resolution microscopy technology (conventional image) and STORMimages acquired in sample picture image generation step S102 isdescribed below while referencing FIG. 4 to FIG. 6. FIG. 4 is a drawingthat compare an image observed without the use of super-resolutionmicroscopy technology (conventional image) and a STORM image acquired inthe sample picture image generation step S102. Part (a) of FIG. 4illustrates an example of a conventional image, and part (b) of FIG. 4illustrates an example of a STORM image. FIG. 5 is a schematic drawingconceptually illustrating a conventional image, and FIG. 6 is aschematic drawing conceptually illustrating a STORM image.

As illustrated in FIG. 4(a), in the conventional image, fine portionsbeyond the optical resolution limit cannot be imaged. As illustrated inFIG. 5, the detection target shown via conventional imaging is shown asa continuous line.

However, as illustrated in FIG. 4(b), the STORM image acquired via STORMimage processing is an image displaying detected results at a resolutionhigher than that in the conventional image. As illustrated in FIG. 6,discretely distributed bright spots are shown.

In order to solve the problems described above, with the microscopedevice 10 according to the present embodiments, after the sample pictureimage generation step S102, analysis processing is performed on thebasis on the STORM image (sample picture image) in the sample pictureimage analysis step S103. For example, on the basis of the positioninformation computed as the positions of bright spots in the STORMimage, the area setting unit sets predetermined areas that contain thepositions of said bright spots as areas corresponding to said brightspots.

Principals Common Throughout the Embodiments

The principals for performing the analysis from the STORM image (samplepicture image) in the present embodiments are described below whilereferencing FIG. 6 described above.

Bright spot P1 and bright spot P2 are shown in FIG. 6. It would beunderstood to a human observer of the sample picture image that thebright spot P1 and the bright spot P2 are in a relationship adjacent toeach other.

However, from the position information (coordinate positions) of each ofthe bright spot P1 and the bright spot P2, one can only discern that thebright spots are disposed so as to be separated a certain distance fromeach other. In some cases, the bright spot P1 and the bright spot P2showing in the sample picture image may not be objects detected as spotsin the same fluorescent image, but may show, in the sample pictureimage, as points in a line which were derived from data detected atdifferent times.

In the event that each of these spots had moved, it would be difficultto determine from the data whether the spot detected first had moved, adifferent spot had moved or, moreover, if the second detected spot was anew spot altogether.

In order to solve this problem, the following processing is performed inorder on each of the bright spots in the sample picture image generatedby STORM. Thereby, predetermined areas C1 and C2 respectivelycorresponding to the bright spots P1 and P2 are associated with thebright spots P1 and P2. Although not illustrated in the drawings, theassociation of the areas is performed on all of the bright spots in thesample picture image.

First, a specific bright spot is established as a reference point.Hereinafter, the phrase “near the bright spot” means “a range withinprescribed conditions, referenced from the position of the bright spot”.For example, the range of “near the bright spot” may be defined on thebasis of the position of the bright spot in space. In this case, therange may be identified as a predetermined area determined by distancefrom the bright spot. Accordingly, in a case where the predeterminedarea is defined as a sphere, the diameter of the sphere, from 10 nm to200 nm, for example, is set as the range.

In other words, the predetermined range can be defined as a spatialwindow having a range from the position of the reference bright spot toa prescribed predetermined distance.

Additionally, “near the bright spot” may, for example, be defined interms of an interval of time from the moment of detection of a brightspot. In this case, if the moment of detection of the bright spot isdefined as the point of origin (starting point) on the time axis, theinterval of time from the moment of detection of the bright spot can beidentified as the time-lapse from the starting point. Additionally, therange of the time-lapse that determines “near the bright spot” can bedefined as a “period”. “Near the bright spot” is preferably set, forexample, as a range from a period of 1 ms from the detection of thebright spot to a period of 10 ms from the same. In other words, it canbe said that with regards to the aforementioned period, a time window isset having an effective detection range starting with the detection of aparticular bright spot and ending upon the passage of the aforementionedperiod.

Within this period, a plurality of STORM fluorescent images aredetected, and at least one sample picture image is generated on thebasis of the fluorescent images in this period.

A value will be identified that represents the state of the spatial area(two-dimensional, three-dimensional) determined by the predeterminedarea in space and the period on the time axis as previously defined. Byidentifying a value that represents the state of the defined spatialarea, a quantitative comparison can be carried out, even in cases whereresults are detected based on differing states. Examples of theaforementioned differing states includes cases where the range ofdetection includes the same predetermined area in space and the samepredetermined period on the time axis, but differing spectral bands;cases where detection is carried out in a different predetermined areain space; cases where detection is carried out in a different period onthe time axis; and the like. Additionally, through this method ofquantitative comparison, for example, the difference in integratedvalues detected on the basis of differing states can be found, or ratioscan be found. As a result, a concrete evaluation value can be computed.In cases where this evaluation value is a predetermined range determinedspatially or temporally, even when the integrated values are detectedunder differing conditions (e.g. position, time, wavelength, and thelike), in cases where the difference from the information computed fromthe previously detected sample picture image is small, it can bepresumed that the same sample has been detected.

As such, by defining the predetermined areas based on the bright spotsin the sample picture image as described above, analysis processingbased on the specified image information can be performed.

Next, conditions common throughout the description of the embodimentsbelow are set forth while referencing FIG. 7.

FIG. 7 is a drawing illustrating bright spots in a STORM image. Thetarget ranges shown in FIG. 7 is explained as showing three-dimensionalspace. However, this should not be construed as a limitation toprocessing on a two-dimensional plane.

The two drawings of parts (a) and (b) in FIG. 7 show sample pictureimages detected under the following states: the same predetermined areain space, the same period, and differing spectral bands of light. Forexample, FIG. 7(a) shows an area where the wavelength band of light ofthe sample picture image SPa is greater than or equal to 530 nm or, inother words, an area where the wavelength is longer than that of greenlight; and FIG. 7(b) shows an area where the wavelength band of light ofthe sample picture image SPb is less than 530 nm or, in other words, anarea where the wavelength is shorter than that of green light. In thefollowing description as well, in each drawing described, 530 nm is setas the threshold and part (a) shows an area where the wavelength band oflight of the sample picture image SPa is greater than or equal to 530nm, and part (b) shows an area where the wavelength band of light of thesample picture image SPb is less than 530 nm.

Positions (coordinates) indicated by reference characters OR1 to OR10 inFIG. 7(a) show the positions (coordinates) of each individual brightspot. Reference characters CR1 to CR10 indicate predetermined areas inspace based on each individual bright spot OR1 to OR10. FIG. 7(a) showsa case where the predetermined areas are shown as spheres (circles).Additionally, positions indicated by reference characters OG1 to OG11 inFIG. 7(b) show the positions of each individual bright spot. Referencecharacters CG1 to CG11 indicate predetermined areas in space based oneach individual bright spot OG1 to OG11. FIG. 7(b) shows a case wherethe predetermined areas are shown as spheres (circles).

In a comparison of part (a) and part (b) in FIG. 7, it is clear that thenumber and position of the detected bright spots is different.Furthermore, in parts (a) and (b) of FIG. 7, due to the differing of theposition of each of the bright spots, it is clear that the distancebetween the bright spots, detected under the same states with theexception of the wavelengths, also differs.

As shown in FIG. 7(a) and FIG. 7(b), the sample picture image analysisunit 143 (FIG. 1) sets predetermined areas containing the positions ofthe bright spots as areas in accordance with the bright spots,corresponding to the positions of the bright spots in each samplepicture image. Position information (coordinate information) of thebright spot and information showing the state of the bright spot areassociated with each of the bright spots and stored in storage unit 16.The sample picture image analysis unit 143 sets a value based on theinformation showing the state of the bright spot to be the valuerepresenting the state of the predetermined area.

The sample picture image analysis unit 143 treats the area within aprescribed radius, having the position shown by the position informationas the center, as the predetermined area. For example, as described inthe following embodiments, the predetermined area described above isassociated with a plurality of unit cells that are smaller than the sizeof the predetermined area. As a result, it is possible to perform ananalysis at a resolution determined by the size of the unit cells. Thesample picture image analysis unit 143 may apply Ratio, FARP, or otheranalysis methods as the analysis at a resolution determined by the sizeof the unit cells.

Hereinafter, different aspects of the invention will be listed in orderas individual embodiments, but the conditions shown in the samplepicture images described above shall be construed to be commonthroughout the embodiments.

First Embodiment

An aspect of the present embodiment, in which areas corresponding tobright spots are set based on positions of the bright spots in a STORMimage, will be described while referencing FIG. 8.

FIG. 8 is a drawing illustrating a method of setting areas correspondingto bright spots in a STORM image in the present embodiment.

In the present embodiment, a sample picture image is analyzed inaccordance with the rules described below as setting method 1.

(Setting Method 1: Case where Predetermined Areas are Filled in withPredetermined Values (Uniformly Filling in Spheres with a ValueRepresenting the State of a Single Bright Spot in Each Sphere))

In the present embodiment, a case is described where a predeterminedvalue is allocated to each of the predetermined areas.

Rule 1: First, a prescribed predetermined constant is set as a valuethat represents the state of a single bright spot associated with eachpredetermined area (sphere). For example, the intensity (number ofphotons) of the fluorescent image forming the bright spot or, rather, anumerical value proportional to the number of photons of the bright spotmay be set as the value representing the state of the bright spot.Rule 2: The value representing the state of the single bright spot,associated with each predetermined area (sphere), is allocated to a setarea within each of the spheres shown as a predetermined area.Rule 3: Next, in cases where there is a plurality of the predeterminedareas (spheres) and the predetermined areas (spheres) have anoverlapping portion, the sum of the numerical values allocated to eachof the overlapping predetermined areas (spheres) in Rule 2 iscalculated, and the resulting sum is newly allocated as the value of theoverlapping area of the predetermined areas (spheres).

In part (a) of FIG. 8, area Z1 a, marked by hatching angled down and tothe left, is an area where two or more predetermined areas (spheres) ofspheres overlap. In part (b) of FIG. 8, area Z1 b, marked by hatchingangled down and to the right, is an area where two or more predeterminedareas (spheres) of spheres overlap. In a comparison of the area Z1 a ofFIG. 8(a) and the area Z1 b of FIG. 8(b), it is clear that the positionsof each of the areas differ, thus showing that quantitative analysis canbe performed easier.

According to these rules, given that the number of photons of the brightspots is uniform, the size of the area where the predetermined areas(sphere) overlap will appear different, depending on the distancebetween the bright spots. In other words, the value allocated to eacharea will vary according to the density at which the bright spots arepresent.

By carrying out processing in accordance with this rule, space can beprocessed as a predetermined area (sphere) of desired size. Desiredresolution information can be acquired by dividing space into smallerpieces. However, due to the amount of data increasing with the number ofdivisions, this processing is suited for cases where the area, oranalysis target, is comparatively narrow.

For example, when analyzing a sample picture image in real-time in whichthe presence of two types of molecules varies from moment to moment,grouping in a range having a diameter of 50 nm is preferable.

Second Embodiment

An aspect of the present embodiment, in which areas corresponding tobright spots are set based on positions of the bright spots in a STORMimage, will be described while referencing FIG. 9.

FIG. 9 is a drawing illustrating a method of setting areas correspondingto bright spots in a STORM image in the present embodiment.

In the present embodiment, a sample picture image is analyzed inaccordance with the rules described below as setting method 2.

(Setting Method 2: Case where Predetermined Areas (Spheres) are Filledin with Predetermined Values (Uniformly Filling in the Spheres with theNumber of Bright Spots Contained in the Spheres))

In the present embodiment, a case is described where a predeterminedvalue, in accordance with the number of the bright spots containedwithin each of the predetermined areas, is allocated.

Rule 1: First, a value representing the state of the bright spotscontained in the predetermined area (sphere) is computed.

For example, a numerical value in accordance with the number of brightspots contained in the predetermined area (sphere) may be a value thatrepresents the state of the bright spots contained in the predeterminedarea (sphere).

Rule 2: The value computed in Rule 1 is allocated to a set area withineach sphere shown as a predetermined area.Rule 3: Next, in cases where there is a plurality of the predeterminedareas (spheres) and the predetermined areas (spheres) have anoverlapping portion, the sum of the numerical values allocated to eachof the overlapping predetermined areas (spheres) in Rule 2 iscalculated, and the resulting sum is newly allocated as the value of theoverlapping area of the predetermined areas (spheres).

In part (a) of FIG. 9, the areas showing three types of differentshading (Z21 a to Z24 a) are areas where the values allocated to thepredetermined areas (spheres) differ from each other. In part (b) ofFIG. 9, the areas showing three types of hatching (Z21 b to Z23 b) areareas where the values allocated to the predetermined areas (spheres)differ from each other. In a comparison of part (a) and part (b) in FIG.9, it is clear that the positions of each of the areas having theirrespective types of shading differ, thus showing that quantitativeanalysis can be performed easier.

According to these rules, the size of each of the predetermined areascan be made uniform, and the density of the bright spots in apredetermined area (sphere) can be expressed by the number of the brightspots contained in that area. For example, even in a case where theflickering time of the bright spots is varied, the impact caused by thevariation in the flickering time can be mitigated, and thus quantitativeanalysis can be carried out, the variation in flickering time havingbeen eliminated.

Third Embodiment

An aspect of the present embodiment, in which areas corresponding tobright spots are set based on positions of the bright spots in a STORMimage, will be described while referencing FIG. 10.

FIG. 10 is a drawing illustrating a method of setting areascorresponding to bright spots in a STORM image in the presentembodiment.

In the present embodiment, a sample picture image is analyzed inaccordance with the rules described below as setting method 3.

(Setting Method 3: Case where a Predetermined Area is Divided into Cubesand Filled in with Predetermined Values (Uniformly Filling in the Cubeswith Values Representing the States of the Bright Spots Contained in theCubes))

In the present embodiment, a case is described where a predeterminedvalue is allocated to each of the predetermined areas.

Rule 1: First, the area is divided by disposing cubes of a predeterminedsize.

Note that the cubes may be disposed at positions determined by a gridformed along orthogonal coordinates. The areas shown as divided cubesare defined as predetermined areas constituting an analysis unit. Notethat by forming a grid along orthogonal coordinates, and making theintervals of the grid equal, the predetermined areas, shown as unitcells, will become cubes. For example, when analyzing a sample pictureimage (three-dimensional image) in real-time in which the presence oftwo types of molecules varies from moment to moment, the unit cellsdescribed above are configured to be cubes, a size thereof defined by alength of one edge being 50 nm.

Note that the grid need not have equal intervals and that thepredetermined area may be configured to have any desired shape (randomshape).

Additionally, the value that represents the state of the predeterminedarea is a prescribed predetermined constant. For example, the value thatrepresents the state of the predetermined area may be a numerical valueproportional to the number of photons of the bright spots (single brightspot or multiple bright spots) contained in the predetermined area or,alternatively, may be a numerical value proportional to the number ofbright spots (single bright spot or multiple bright spots) contained inthe predetermined area.

Rule 2: The values representing the states of the predetermined areas(cubes), are allocated to each set area shown as a predetermined area.

Note that in FIG. 10, results through Rule 2 are shown as obliquedrawings.

In part (a) of FIG. 10, cubes labeled with reference numerals QR1 toQR10 are disposed in areas of a part of the grid. The position of eachof the cubes QR1 to QR10 is determined according to the position of thebright spots having a wavelength within a specific range. The specificwavelength range in FIG. 10(a) is, for example, wavelengths greater thanor equal to 530 nm. Here, according to the rules described above,predetermined numeric values are allocated to the cubes QR1 to QR10 inaccordance with each of the bright spots, as the values representing thestates of the predetermined areas. For example, the numeric values forthe cubes QR1 to QR10 may be set, in order, as 2, 2, 3, 3, 3, 3, 2, 1,1, and 1, respectively.

In part (b) of FIG. 10, cubes labeled with reference numerals QG1 toQG11 are disposed in areas of a part of the grid. The position of eachof the cubes QG1 to QG11 is determined according to the position of thebright spots having a wavelength within a specific range. The specificwavelength range in FIG. 10(b) is, for example, wavelengths less than530 nm. Here, according to the rules described above, predeterminednumeric values are allocated to the cubes QG1 to QG11 in accordance witheach of the bright spots, as the values representing the states of thepredetermined areas. For example, the numeric values for the cubes QG1to QG11 may be set, in order, as 2, 3, 3, 3, 3, 3, 2, 4, 3, 3, and 1,respectively.

In a comparison of part (a) and part (b) in FIG. 10, because theposition of the area shown by each cube is determined according to thewavelength of the bright spot present in the unit cell, it is clear thatthe position of each cube differs according to the wavelength of thebright spot, and, it is understood that this facilitates the performanceof quantitative analyses.

In accordance with these rules, the size of each of the predeterminedareas is made uniform as the size of the unit cell or, rather, the cube.Thus, the value in accordance with the number of photons of the brightspots (single bright spot or multiple bright spots) assigned within thisarea or, alternatively, the numerical value proportional to the numberof bright spots (single bright spot or multiple bright spots) containedin the predetermined area can be set.

By carrying out processing in accordance with this rule, space can beprocessed as a predetermined area (cube) of desired size. Desiredresolution information can be acquired by dividing space into smallerpieces. However, due to the amount of data increasing with the number ofdivisions, this processing is suited for cases where the area, oranalysis target, is comparatively narrow.

Additionally, in a comparison with conventional confocal images, datathat is readily comparable can be generated by combining the pitch ofthe pixels of the confocal image with the pitch at which the cubesdescribed above are disposed.

In the present embodiment, attention is paid to the fact that, in thegenerated information, the original signal shape (sphere) is replacedwith a cube.

Fourth Embodiment

An aspect of the present embodiment, in which areas corresponding tobright spots are set based on positions of the bright spots in a STORMimage, will be described while referencing FIG. 11.

FIG. 11 is a drawing illustrating a method of setting areascorresponding to bright spots in a STORM image in the presentembodiment.

In the present embodiment, a sample picture image is analyzed inaccordance with the rules described below as setting method 4.

(Setting Method 4: Case where a Determination Area (Sphere) andPredetermined Areas (Cubes) are Defined, and the Values of thePredetermined Areas are Set to a Value Based on Results Determined bythe Determination Area (Sphere).

In the present embodiment, a case is described where a predeterminedvalue is allocated to each of the predetermined areas.

Rule 1: First, the analysis target range is divided into areas based ona grid having preset intervals. Areas shown as divided unit cells aredefined as the predetermined areas, each of which constituting ananalysis unit.

Note that by forming a grid along orthogonal coordinates, and making theintervals of the grid equal, the predetermined areas, shown as unitcells, will become cubes. For example, when analyzing a sample pictureimage (three-dimensional image) in real-time in which the presence oftwo types of molecules varies from moment to moment, the unit cellsdescribed above are configured to be cubes, a size thereof defined by alength of one edge being 50 nm.

Rule 2: Next, the determination areas (spheres) corresponding to eachbright spot are specified. The radius of the determination areas is setso that the determination areas have a size which is greater than thesize of the unit cells (cubes).

Referencing the size of the unit cells described above (cubes, where thelength of one side is 50 nm), the size of the determination areas may,for example, set so that the diameter thereof is 200 nm (radius: 100nm). Because the size of the unit cell and the size of the determinationarea are set individually, the size of the unit cell and the size of thedetermination area can easily be set. For example, by setting a spherewith a radius in accordance with the number of photons as thedetermination area, even if a case arises where the size of the unitcell becomes greater than the size of the sphere, the analysisprocessing described below can be carried out without the need to adjustthe size of the determination area.

Rule 3: Next, a value based on information showing the state of thebright spot positioned at the center of each of the determination areas(spheres) is allocated to each of the determination areas (each sphere).For example, a value proportional to the number of photons of the brightspot may be associated with the information showing the state of thebright spot.Rule 4: Next, the value in accordance with the determination area(sphere) contained in each of the unit cells (cubes) is set as the valueshowing the state of the bright spot contained in the unit cell. Forexample, the number of photons dependent on a ratio of the volume of thedetermination area (sphere) contained in the unit cell (cube) to thevolume of the determination area (sphere) may be set as a constant valuein the cube as the value in accordance with the determination area(sphere) contained in the unit cell (cube).Rule 5: Next, in cases where the areas constituting the plurality ofdetermination areas (spheres) have a portion where the determinationareas (spheres) overlap, the sum of the numerical values allocated toeach of the overlapping determination areas (spheres) in Rule 4 iscalculated, and the resulting sum is set as the value showing the stateof the bright spot contained in the unit cell (cube).

Note that in FIG. 11, results through Rule 4 are shown as obliquedrawings, and the results of Rule 5 are omitted.

By carrying out processing in accordance with this rule, space can beprocessed as a unit cell (cube) of desired size. Desired resolutioninformation can be acquired by dividing space into smaller pieces.Additionally, when analyzing sample picture images (data) havingdifferent spectra (wavelengths), by using the unit cell (cube) describedabove, positions in space can be standardized and, as a result, analysisis facilitated. Note that in the present embodiment, the determinationareas (spheres) and the predetermined areas (cubes) are setindividually, but a configuration is possible wherein only predeterminedareas of any cubic shape are set and the numerical value in accordancewith the number of bright spots contained in each predetermined area isset as the value that represents the state of the bright spots containedin the predetermined area.

Fifth Embodiment

An aspect of the present embodiment, in which areas corresponding tobright spots are set based on positions of the bright spots in a STORMimage, will be described while referencing FIG. 12.

FIG. 12 is a drawing illustrating a method of setting areascorresponding to bright spots in a STORM image in the presentembodiment.

In the present embodiment, a sample picture image is analyzed inaccordance with the rules described below as setting method 5.

(Setting Method 5: Case where the Size of the Predetermined Areas isDetermined According to the Density of the Bright Spots, and thePredetermined Areas are Filled in with Predetermined Values (SpheresUniformly Filled in) in Accordance with Information Representing theStates of the Bright Spots)

In the present embodiment, a case is described where a predeterminedvalue is allocated to each of the predetermined areas.

The case of the present embodiment differs from the case illustrated inpart (a) of FIG. 7 where the spheres CR1 to CR10 have uniform radii. Inpart (a) of FIG. 12 of the present embodiment, spheres CR1′ to CR10′ areshown which have radii that differ based on the positions of the brightspots OR1 to OR10. Part (b) of FIG. 12 differs from the case illustratedin part (b) of FIG. 7 where the spheres CG1 to CG11 have uniform radii.In part (b) of FIG. 12 of the present embodiment, spheres CG1′ to CG11′are shown which have radii that differ based on the positions of thebright spots OG1 to OG11. The spheres are generated in accordance withthe following rules.

Rule 1: First, a sphere is detected having, as its radius, a distancefrom a first bright spot (reference) to an adjacent second bright spot(closest to the first bright spot). The detected sphere is set as apredetermined area corresponding to the first bright spot. A valuerepresenting the state of the bright spot is set in advance for thesingle bright spot associated with each predetermined area (sphere). Forexample, a numerical value proportional to the number of photons of thebright spot may be set as the value representing the state of the brightspot.Rule 2: Values calculated from the value representing the state of eachof the bright spots and the volume of the predetermined areas (spheres)are allocated to each of the areas in the spheres shown as predeterminedareas. For example, the result of dividing the value based on theinformation showing the state of the first bright spot by the valueshowing the size (volume) occupied by a first area may be set as thevalue representing the state of the bright spot contained in the firstarea.

Note that in cases where there is a plurality of predetermined areas(spheres) and the predetermined areas (spheres) have an overlappingportion, the sum of the numerical values allocated to each of theoverlapping predetermined areas (spheres) in Rule 2 may be calculated,and the resulting sum may be newly allocated as the value of theoverlapping area of the predetermined areas (spheres).

Following the rules described above, the size of each of thepredetermined areas is set according to the density of the bright spotsthat are present therein, and the number of photons of the bright spotscan be distributed in accordance with said density. In other words, thevalue allocated to each region can be made to vary in accordance withthe density at which the bright spots are present.

By carrying out processing in accordance with these rules, apredetermined area (sphere), of a size set in accordance with thedensity at which the bright spots are present, can be set and processingcarried out. Resolution can be increased in cases where the density atwhich the bright spots are present is high, and resolution can bedecreased in cases where the density at which the bright spots arepresent is low. As described above, resolution can be adjusted on thebasis of the density at which the bright spots are present and, as aresult, the calculation processing load can be adjusted according to thedensity at which the bright spots are present and analyses can beefficiently carried out.

As shown in the embodiments, the microscope device 10 (analysis device)can perform quantitative analysis based on images of high resolution.

Although only a few embodiments of the present invention have beendescribed, it should be understood that the present invention may beembodied in many other specific forms without departing from the spiritor the scope of the present invention.

For example, in the embodiments, cases are described where STORM isgiven as the type of fluorescent microscope device used to acquireimages of super resolution and quantitative analyses based on imagesacquired using these methods are carried out. However, the presentinvention also can be applied to cases where quantitative analysis iscarried out on images (sample picture images) captured usingphotoactivation localization microscopy (PALM) technology such as thatdescribed in U.S. Pat. No. 7,626,695 and the like.

An example of the fluorescent material used when employing PALM isDronpa. Dronpa has properties whereby when irradiated with light of agiven intensity, Dronpa is enabled to absorb the excitation wavelength.Additionally, Dronpa does not emit fluorescence while in an inactivestate, even when irradiated with excitation light. Accordingly, just asin STORM, low-resolution fluorescent images and high-resolutionfluorescent images can be acquired by adjusting the intensity of lightradiated on the sample. By using the low-resolution fluorescent imagesas mask images, quantitative analysis can be carried out on thehigh-resolution images.

Alternatively, the present invention also can be applied to cases wherequantitative analysis is carried out on images (sample picture images)captured using stimulated emission depletion (STED) microscopy, in whichimages are observed by irradiating the optical system of a galvanometermirror-based confocal microscope with two types of laserlight—excitation laser light for observation and short pulse laser lightfor inducing emission—nearly simultaneously. In this case as well,images according to the embodiments of the present invention or imagesacquired via structured illumination microscopy (SIM) as described abovecan be used as a conventional image. Note that various aspects of theembodiments described above may be combined as appropriate. Moreover,some of the component parts may be removed. Moreover, to the extentpermissible by law, all publications and US patent documents related tothe devices or the like used in the embodiments and modificationexamples as described above are incorporated herein by reference.

REFERENCE SIGNS LIST

-   10 Microscope device (analysis device)-   11 Excitation illumination system-   13 Activation illumination system-   14 Control unit-   15 Microscope main body-   16 Storage unit-   17 Display unit-   21, 42 Laser light source-   22 Shutter-   31 Stage-   32 Total reflection mirror-   33 Dichroic mirror-   34 Camera-   43 Scanner-   61 Mercury lamp-   141 Initialization processing unit-   142 Picture image forming unit-   143 Sample picture image analysis unit-   L1 Excitation light-   L2 Activation light-   S101 Initialization process step-   S102 Sample picture image generation step-   S103 Sample picture image analysis step-   G1 Conventional image-   G2 STORM image

What is claimed is:
 1. A device comprising: a processor programmed to:specify, in a fluorescent image, at least a first area and a second areain an analysis target range, and calculate quantitative information ofthe first area based on information regarding a bright spot included inthe first area; calculate quantitative information of the second areabased on information regarding a bright spot included in the secondarea; and calculate quantitative information of an overlapping area inwhich the first and second areas overlap based on the informationregarding the bright spot included in the first area and the informationregarding the bright spot included in the second area.